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SYSTEM AND METHOD FOR LEVERAGING DATA INTO A GAME 



game systems. More partictilarly, the present invention relates to a method 
and system for leveraging data into a game platform. 

BACKGROUND OF THE INVENTION 

Currently^ entertainment systems have limited capabilities to allow a 
viewer to participate in or interact with a sports event broadcast due to the 
limitations of the information provided to the entertainment systems. For 
example^ a viewer of a baseball game or football game broadcast on a television 
may view a graphical enhancement provided by the provider or source of the 
broadcast such as the score of a game or statistics of a team or player. 
However, the program provider controls the graphical enhancement. 
Consequently, a viewer is not able to control the visual enhancement or its 
content. As such, it is difficult for a sports enthusiast to participate in or 
interact with a sports event while viewing the sports event broadcast. 

Furthermore, current game systems have limited capabilities for 
allowing a player or multiple players to participate in or interact with a live 
sports event broadcast. Typically, players of game systems play video sports 
games on a television display in which the displayed content is provided from 
a storage medium such as cartridge, compact disc (CD), or digital video disc 
(DVD). In addition, the type of game to be played is dictated by the content of 
such storage mediums. As such, current game systems do not provide players 
the ability to play selectively a game on a television display correlated with a 
live sports event broadcast. 

One popular form of participation in or interaction with a sports event is 
a "fantasy team" league in which players choose a team of selected players to 
compete against other players based on accumulated points. The points are 
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accumulated in relation to the performance of the selected players during a 
particular sports event. However^ t3^ical providers of such fantasy team 
leagues, for example, web sites, do not allow players to view the point 
standings of the multiple players as events are taking place during the sports 
event broadcast. Furthermore, web sites are unable to allow a player to view 
the points standings and the sports event broadcast at the same time. Also, 
besides player selection for a fantasy team, players are limited in affecting their 
score during the sports event broadcast. 

SUMMARY OF THE I^fVENTION 

A system and method for leveraging data into a game platform are 
described. Supplemental data related to a broadcast event is received. 
Supplemental data is further processed selectively to obtain processed data. 
Finally, the processed data is integrated into a game platform, such that the 
processed data is associated with at least one user controllable game element 
generated by the game platform. 

Other features and advantages of the present invention will be apparent 
from the accompanying dravraigs, and from the detailed description, which 
follows below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example and not limited 

by the figures of the accompanying drawings in which like references indicate 

similar elements and in which: 

Figure 1 illustrates an exemplary interactive environment allowing a 

user to participate with a sports event broadcast- 
Figure 2 illustrates exemplary revenue streams; 
Figure 3 is a block diagram of an exemplary communications network; 
Figure 4 is an internal block diagram of one embodiment of an 

application/ game system; 
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Figure 5 illustrates one embodiment of an entertainment system 
receiving audio-video data and supplemental data; 

Figure 6 illustrates one embodiment of various providers providing 
supplemental data with audio-video data for an entertainment system; 

Figure 7 illustrates one embodiment of a game platform integrated with 
an entertainment system capable of receiving audio-video data and 
supplemental data; 

Figure 8 illustrates another embodiment of a game platform integrated 
with an entertainment system capable of receiving audio-video data and 
supplemental data from a broadcast and information from an auxiliary 
information source; 

Figure 9 is an internal block diagram of one embodiment of a receiver or 
set-top box; 

Figure 10 is a functional block diagram of one embodiment of an 
application/game platform module; 

Figure 11 illustrates one embodiment of an electronic program guide for 
allowing a user to access interactive services; 

Figures 12a through 12h are exemplary display screens showing user 
controllable visual enhancements in participating with a sports event 
broadcast; 

Figure 13 is a block diagram of one embodiment of an application/game 
platform; 

Figure 14 is a flow diagram of the method for leveraging data into a 
game platform; 

Figure 15 is a flow diagram of one embodiment of a procedure for 
processing supplemental data; and 

Figure 16 is a flow diagram of one embodiment of a procedure for 
integrating processed data into the game platform. 
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DETAILED DESCRIPTION 

A system and method for leveraging data into a game platforai is 
described. In the following description, for purposes of explanation, numerous 
specific details are set forth in order to provide a thorough understanding of 
the present invention. It will be evident, however, to one skilled in the art that 
the present invention may be practiced without these specific details. 

In the following description, interactive techniques are described 
allowing a user or viewer or a customer of a service ("user") to interact and 
participate with a sports event broadcast while viewing the sports event 
broadcast. The interactive techniques, however, may be implemented with 
other types of broadcast programs. For example, the interactive techniques 
may be modified to interact and participate with a game show allowing a 
user/player or multiple players to play along with the game show or play each 
other for points. 

Furthermore, in the following description, a sports event broadcast 
includes a live broadcast, pre-recorded broadcast, or a broadcast from a storage 
device such as compact disc (CD) or a digital video disc (DVD). A "live 
broadcast" may also refer to a substantially live broadcast in which there is a 
short delay in the transmission of the broadcast. 

Figure 1 illxistrates an exemplary interactive environment 100 allowing a 
user to participate with a sports event broadcast. Interactive services may be 
provided for interactive environment 100 via an application or game platform 
that allows a user to interact or participate with a sports event broadcast. 

The exemplary interactive services described herein may allow a user to 
interact or participate with a sports event broadcast by controlling a visual 
enhancement on the broadcast, participating in a multi-player fantasy team 
league and viewing results of the league as events take place in the broadcast, 
participating in a virtual type game with the broadcast, or rendering the 
broadcast by creating or reconstructing a visual rendition of the event using 
data or information provided with the broadcast. Such exemplary interactive 
services will be discussed in more detail in the description that follows. 
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For one embodiment, interactive environment 100 includes various 
providers of data and services for a home network having digital television 
capabilities. For an alternate embodiment, interactive environment 100 may 
include various providers of data and services for a home network having 
analog television capabilities. 

Referring to Figure 1, interactive envirorunent 100 includes home 
network 105 coupled with video provider 110 and web provider 114. 
Interactive environment 100 also includes data provider 112 coupled to video 
provider 110 and web provider 114, and application/game provider 116 
coupled with home network 105 via application/game platform 103. 

Home network 105 includes a digital television (DTV) 101 coupled to a 
receiver or set-top box (STB) 102, which may receive inputs from remote 
control 119 or application/game platform 103. In an alternative embodiment, 
the functior>s of receiver 102 may be contained in DTV 101 in which DTV 101 
may receive inputs directly from remote control 119 or application/game 
platform 103. 

DTV 101 may output audio-video data ("A/ V data") of a live or a 
substantially live sports event broadcast and user controllable visual 
enhancements. For one embodiment, the visual enhancements may be 
generated from supplemental data contained in the sports event broadcast. 
Alternatively, the visual enhancements may be generated using both A/ V data 
and supplemental data in the broadcast. 

For example, the A/V data of a live sports event broadcast may be 
displayed on DTV 101 as shown in screen 106. Also, user controllable visual 
enhancements (generated graphical objects) of a virtual race car may be 
displayed on DTV 101 as shown in screen 107 using supplemental data, which 
is "bundled" with the A/V data in the sports event broadcast. Screens 106 and 
screens 107 may be displayed together or separately on DTV 101. Screen 107 
shows a virtual cockpit, which is a rendered cockpit of the race car in screen 
106, and a virtual map indicating the position of the race car in screen 106. 
That is, the virtual cockpit in screen 107 is a rendered visual enhancement of 
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the cockpit of the race car in screen 106, which displays supplemental data such 
as telemetry data taken from the race car in screen 106. For example^ the 
speedometer readings of the virtual cockpit of screen 107 represent the same 
speedometer readings of the actual race car of screen 106. 

Telemetry data may be supplemental data or information that is 
transmitted with the A/ V data representing the live sports event broadcast. In 
an alternative embodiment a pre-recorded sports event broadcast may be 
displayed on DTV 101 along with user controllable visual enhancements, which 
may be derived from supplemental data contained in the pre-recorded sports 
event broadcast. 

STB 102 is a receiver for DTV 101. STB 102 may receive program 
broadcasts from a plurality of program providers or sources. Furthermore/ 
STB 102 may receive supplemental data with the A/V data of the program 
broadcasts. STB 102 may receive A/V data and supplemental data for 
application/ game platform 103 and DTV 101 via a local antenna connection, 
cable connection, or satellite connection. Although DTV 101 is configured to 
receive digital signals, a display unit for receiving analog signals may be 
implemented for home network 105. For one embodiment, DTV 101 may 
include receiver 102 internally without connecting to an external receiver for 
receiving program broadcasts. 

Application/ game platform 103 may be connected to STB 102 via an 
external connection such as an IEEE 1394 connection. Alternatively, 
application/game platform 103 may be connected directly to DTV 101 and may 
be coimected to a network such as the Internet via an Internet Protocol (IP) 
connection. Application/game platform 103 is an interface between a user and 
DTV 101, which allows the user to participate with a sports event broadcast on 
DTV 101. Altematively, application/game platform 103 may be implemented 
in software or hardware or both for providing an interface between multiple 
users and DTV 101. As such, multiple users are able to participate with a 
sports event broadcast on DTV 101 using application/ game platform 103. 
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For one embodiment, application/game platform 103 may be software 
or hardware within STB 102 or DTV 101. For an alternate embodiment, 
application/game platform 103 may be a stand-alone machine such as a SONY 
Playstation™ connected to STB 102 or DTV 101. Application/game platform 
103 includes the necessary hardware or software to process supplemental data 
being provided to home network 105. For one embodiment, application/game 
platform 103 generates user controllable visual enhancements using processed 
supplemented data. For alternate embodiments, application/game platform 
103 may process A/V data and supplemental data to generate user controllable 
visual enhancements. 

For one embodiment, home network 105 provides access to interactive 
services via application/game platform 103. For example, interactive services 
may be provided for a race car sports event broadcast. The interactive services 
may be enabled by the delivery of live A/V data of a race car event and 
supplemental data such as, for example, telemetry data, camera data, and race 
data in digital form to home network 105 via STB 102. A user may control and 
generate visual enhancements or play in a fantasy team or play in a 
synchronized game environment using the supplemental data along with the 
A/V data of the sports event broadcast via application/game platform 103. 

The interactive services provided by application /game platform 103 are 
integrated with the various data feeds to home network 105. For example, 
application/game platform 103 may receive supplemental data to generate 
visual or graphical enhancements on the broadcast being displayed on DTV 



Video provider 110 provides the service of acquiring or capturing A/V 
data of a live sports event. Alternatively, video provider 110 may acquire or 
capture A/V data of a previously recorded sports event. For one embodiment, 
video provider 110 includes or "bvmdles" the A/V data with supplemental data 
such as real-time operational data from a live sports event provided from data 
provider 112. For example, the supplemental data may be based on a race car 
sports event, which includes telemetry data, camera data, and race data of the 
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race car sports event. Video provider 110 also provides the service of 
broadcasting the A/V data with the supplemental data to DTV 101 in home 
network 105 via receiver or STB 102 for a viewer. Video provider 110 may be a 
broadcasting network such as ESPN™ or ABC™. Alternatively, video provider 
may be a cable network such as ATT™ or a satellite network such as 
DirectTV™. 

Data provider 112 provides the service of acquiring the supplemental 
data and providing the supplemental data to video provider 110, v^hich 
bimdles the A/V data vsnth the supplemental data for the sports event 
broadcast. Data provider 112 may also provide supplemental data to web 
provider 114, which also provides a data feed to home environment 105 via an 
IP connection. For example, data provider 112 may be the sponsor of a race car 
sports event such as Championship Auto Racing Teams CART™ that provides 
the service of acquiring real-time data such as telemetry data, camera data, and 
race data associated with the race car event. 

Web Provider 114 provides Internet related services 118 to a user of 
home network 105 via an IP connection. For example, a user may select and 
register for participation in a fantasy team league in connection with a sports 
event broadcast through web services 118. Web services 118 may feed data or 
information to home network 105 via web provider 114. For example, 
application/ game platform 103 may be connected to the Internet and receive 
fantasy team information from web services 118 via web provider 114. 
Application/game platform 103 may process the information from web 
services 118 to allow a user to view results of a fantasy team on DTV 101 as 
events are taking place in a live sports event broadcast. For example, 
application/game platform 103 may generate graphical data to be displayed on 
DTV 101 that lists teams and players and accimnulated points related to the 
teams or players as events are taking place the sports event broadcast. Thus, a 
user may view fantasy team results as events are taking place in a live sports 
event broadcast while viewing the sports event broadcast on the same display. 
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For one embodiment, web provider 118 is responsible for managing the 
cx)mpetition and points accumulation of a fantasy team league and providing 
such information to home network 105. Web provider 118 may also allow a 
user of home network 105 to select teams or change players. Web provider 114 
may provide such information to be displayed through a visual enhancement 
on DTV 101 via STB 102, which may also be connected with web provider 114 
via an IP connection. Thus, a user/ player may view the points accumulation of 
fantasy teams of mtiitiple players as events take place during the sports event 
broadcast. 

Application /Game provider 116 provides interactive applications for a 
user to participate in a sports event within home network 105. For one 
embodiment, application/game provider 116 provides software for 
application/ game platform 103, which provides an interactive application that 
allows graphical elements or objects to be linked to a live sports event 
broadcast For example, a user may receive telemetry data from a race car in a 
race Ccir sports event to be displayed as a visual or graphical enhancement on 
DTV 101. The graphical enhancement represents information contained in the 
telemetry data. 

For another embodiment, application /game provider 116 may provide 
virtual game applications used by application /game platform 103. For 
example, application/game platform 103 may generate a virtual car on a live 
race car sports event broadcast. A user/player may control the virtual car to 
compete against the race car drivers in the actual live broadcast. The virtual 
car can react to other race cars in the live race car sports event broadcast that is 
controlled based on received supplemental data such as telemetry data or 
position data of the other race cars. Alternatively, application/game platform 
103 may render the sports event broadcast using the telemetry data, race car 
data, camera data, and other positioning data to create virtual cars in a virtual 
gaming environment. The virtual cars may be synchronized to the actual race 
cars partidpahng in the live race car sports event broadcast. 



9 



PCTAJSOO/30847 



WO 01/36061 



For purposes of explanation, a race car sports event broadcast is used to 
illustrate the operation of interactive environment 100 allowing a viewer to 
control personalized visual enhancements using supplemental data, which are 



car sports event broadcast. For example, the race car sports event may be 
sponsored by CART™ using "Indy" like race cars. CART™ races are broadcast 
by network providers such as ESPN™ or ABC™. Alternatively, interactive 
environment 100 may be modified to allow a user to interact and participate 
with other sport events such as a football game, basketball game, soccer game, 
other similar sports events in which similar types of supplemental data are 
being provided. 

For one embodiment, to allow a user to interact and participate with a 
race car event, race cars in the sports event are equipped with sensor devices 
that measure and transmit supplemental data such as real-time operational 
data or "telemetry data." For example, telemetry data may include car velocity, 
gear position, rpm, braking, track position, trar\sverse force and other real-time 
operational data. The telemetry data may be acquired by pit crews and 
broadcasters to monitor car and driver performances. The telemetry data may 
be included or ("bimdled") with the audio-video data or A/ V data of the 
broadcast that allows a viewer of the broadcast to generate and control visual 
enhancements with the race car sports event broadcast. For example, a video 
enhancement may be shown on the race car sports event broadcast showing a 
graphical speedometer indicating the actual speed of a race car in the 
broadcast. 

Furthermore, other types of supplemental data such as camera data and 
race data may also be bundled with the A/ V data of a broadcast for a viewer to 
generate visual eiUiancements. Camera data may identify camera positiorts, 
orientation, field-of-view (zoom), focus, or radial distortion coefficients firom a 
plurality of cameras. Race data may include standings, car status, number of 
laps remaining for each race car driver. By bimdling A/ V data with 
supplemental data, a number of interactive services are capable of being 



supplied by video provider 110 or web provider 114 while tuning into a race 
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provided to a user or customer of the interactive services. For example, a user 
may select camera data from one of the plurality of cameras to render a view of 
the data from the camera as a graphical enhancement on DTV 101. 
Furthermore, a user may render the broadcast by creating virtual graphical or 
visual enhancements of race cars that are being captured by the cameras in a 
virtual environment. 

Thus, because such supplemental data is capable of being provided 
along A/V data of the race car sports event broadcast, interactive envirorunent 
100 is capable of using the supplemental data to allow a viewer or customer of 
an interactive service to request a type of visual enhancement to be displayed 
allowing the user to interact and participate with the race car sports event. For 
example, STB 102 may, upon a user request, separate received supplemental 
data and provide the supplemental data to application/game platform 103 that 
processes the data to generate visual enhancements on DTV 101. For one 
embodiment, a user is able to control application/game platform 103 using 
remote control 119 or other types of inputs to control and generate visual 
enhancements on DTV 101. 

The visual enhancements that may be created are based on a type of 
interactive service requested and a type of supplemental data being provided. 
The interactive services described herein allow a user to participate with a 
sports event and view the sports event at the same time. Furthermore, the 
interactive services allow a user to control personalized visual enhancements 
while tuning into the auto race broadcast. That is, a user controls the level of 
enhancements for the auto radng sports event and the level of interaction with 
the sports event broadcast. 

For example, a sports enthusiast viewer may request in-depth 
performance related enhancements while a beginner sports fan may request 
basic level erJiancements. Furthermore, the interactive services may provide a 
game environment, in which a viewer may play other viewers, for example, in 
a fantasy team league setting. 
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The following are exemplary categories of interactive services that 
provide varying types of enhancements for interactive environment 100. The 
exemplary categories are: Controlling a visual enhancement in the broadcast. 
Participating in a multi-player fantasy team league and viewing results of the 
league as events take place in the broadcast. Participating in a virtual type 
game with the broadcast, and Rendering a broadcast by creating or 
reconstructing a visual rendition of the sports event using data or information 
provided vn\h the broadcast. Exemplary types of interactive services for the 
above exemplary categories are described below. 



In-car video with animated cockpit: a user may request that a virtual and 
graphical enhancement of a cockpit with gauges of a selected car in the actual 
racing event be displayed or superimposed on the audio-video data of the race 
car broadcast. Thus, a user may view the actual measurements of the gauges in 
the actual race car from the virtual enhancements of the gauges. 

Map: a user may request a visual enhancement of a rendered track with 
icons showing current positions of selected race cars in the actual race car 



On-demand replays: a user may select or request that a favorite scene (e.g., 
a crash or a pit stop) be saved in a local storage in which the favorite scene may 
be replayed at a later time. The local storage may be contained in a home 
network environment or in a remote location such as a database connected to 
the Internet. 



Selection and comparison of fantasy teams: a user may request a visual 
enhancement to build and select a fantasy team of drivers. Multiple users may 
participate in which point standings of the teams may be displayed in a visual 
enhancement. The visual enhancement may show updated points as events are 
taking place in a sports event broadcast. 



Control of Visual Enhancements 



event. 



Participation in a Fantasy Team 
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Current standings and team performance: a user may request a visual 
enhancement to display selectively current standings, lap times, and/or other 
information for racers of selected fantasy tean\s. 

Visual comparison of driver performance and technique: a user may request a 
visual enhancement to select performance parameters (e.g., acceleration around 
a curve) and compare the performance parameters of other drivers. 

Interactive polls: a user my request a visual enhancement of trivia 
questions or questions related to the sports event to earn additional points for a 
selected fantasy team. For example, a user may be presented with a trivia 
question and a list of possible answers in a visual enhancement related to the 
sports event. 

Rendered Broadcast 

Rendered Environment: a user may request to render a broadcast such 
that a visual rendition of the event is created or reconstructed using data or 
information provided with the broadcast. For example, real-time operational 
data such as telemetry data of actual race cars in a radng event may feed into a 
home network. The real-time operation data may then be processed by the 
home network to create virtual elements or objects to reconstruct or recreate 
the broadcast in a rendered environment or virtual world. The rendered 
environment may be shown separately or together on a display with the sports 
event broadcast. Alternatively, the rendered environment may be sent to a 
remote display. Such rendered environments may have applications for virtual 
games. 

FarticipatiDn in a Virtual Game 

Live Composite Game: a user may request to play and control a virtual car, 
which is superimposed on the audio-video data of a live race car sports event 
broadcast. The virtual car may react to the actual race cars in the broadcast 
such that an application/game platform uses real-time operation data such as 
telemetry data associated with the actual race cars to cause the reaction to the 
virtual car. Alternatively, a user may be connected to a game environment in 
which real-time operation data of the actual race cars in a broadcast feed into 
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the game environment, and the game environment recreates the broadcast in a 
virtual game world. 

The above enhancements will be discussed in more detail below with 
reference to Figures 12a through 12h. Furthermore, the above enhancements 
are exemplary in nature in which other types of enhancements nuiy be used. 

Figure 2 depicts an illustration 200 showing exemplary revenue streams 
related to interactive services that allow participation with a sports event 
broadcast. Referring to Figure 2, revenue streams 221 may include: a Pay Per 
View (PPV) revenue stream 222, Pay Per Play (PPP) revenue stream 223, DTV 
Premium Service revenue stream 223, Advertising revenue stream 224, 
Sponsorship revenue stream 226, Game Sales/Updates revenue stream 227, 
Subscription revenue stream 228, and Web Based revenue stream 229. A data 
or a service provider may be a recipient of the revenue streams. The data or 
service provider may be the same entity or be different entities. Furthermore, 
cost and revenue sharing may be negotiated by varying entities. 

For a pay per view (PPV) revenue stream, a user or customer 
("customer") purchases a PPV interactive service related to a particular sp>orts 
event broadcast. For example, a user may purchase an interactive service from 
a video provider such as ABC™ and an authorized data provider such as 
CART™ to provide the A/V data and supplemental data for a race car event. 
The video provider and data provider provides the A/V data and 
supplemental data such that customers who purchase the service are able to 
receive the A/V data and supplemental data. The interactive service provider 
may allow the customer to receive A/V data and supplemental data, for 
example, via application/game platform 103, which may process the A/V data 
and supplemental data and generate user controllable visual enhancements on 
the broadcast. 

For a pay per play (PPP) revenue stream, a customer purchases an 
interactive service that may be monitored at the home envirorunent. The 
customer is charged a monthly fee based on the amount of time or number of 
times the service is used. For example, an interactive service provided through 
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a receiver for a TV may be connected with an external monitoring system. The 
receiver may be connected via a telephone line to a system that monitors usage 
of the service in the receiver. 

For a DTV premium service revenue stream, a customer purchases a 
service designed for digital televisions. For example, a customer may purchase 
a DTV premium service from a satellite provider that provides interactive 
services designed for a DTV. 

For an advertising revenue stream, providers of interactive services may 
charge for advertising space on any visual enhancements being displayed by a 
user. For example, a company advertisement may be displayed on a visual 
enhancement while the customer is viewing a sports event broadcast. 

For a sponsorship revenue stream, a sports event sponsor such as 
CART™ may charge for the services of gathering real-time operational data 
such as telemetry data to allow customers to participate in a CART™ sports 
event broadcast. For example, CART™ may charge a fee to a broadcasting 
network such as ABC™ for such services. 

For a game sales/updates revenue stream, a customer purchases a 
CD/DVD having applications stored thereon to generate user controllable 
visual enhancements using audio/video data and supplemental data. 
Alternatively, a customer may purchase such applications via an online 
download. 

For a subscription revenue stream, a customer may purchase a 
subscription, for example, from a cable company to provide interactive services 
to allow a user to participate with a sports event broadcast. 

For a web based revenue stream, a customer may be charged a fee by 
web providers providing web services that provide data and information for 
customers to select and participate in a fantasy team league competition. 

The running costs of an interactive service are primarily based on 
acquisition and broadcast of video content, acquisition and delivery of real- 
time data, and web service operation including viewer incentives such as 
fantasy team prices and virtual leagues. As illustrated, the sources of revenue 
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may be video and web advertising, PPV, PPP, subscription (e.g., fantasy team 
registration). The points owned by a player/viewer and maintained by the 
web provider provide additional revenue opportunities. In addition, a tiered 
pricing model may be used. For example, the primary service (the video 
broadcast) could be offered for no charge while the enhanced service would be 
charged (whether by PPV, PPP or subscription). The enhanced service could be 
further divided into interactive and gaming components, which could be 
offered separately or in combination. 

Figure 3 is a block diagram of one embodiment of a communications 
network 300. Communications network 300 illustrates an exemplary 
interconnected network for interactive environment 100 of Figure 1. 

Referring to Figure 3, a plurality of computer systems are shown in the 
form of M servers (servers 330-1 through 330-M) and N clients (clients 340-1 
through 340-N), which are coupled to each other via network 350, terrestrial 
based wireless communications links 360-1 through 360-T. The M servers and 
N clients may also be coupled to each other via space based communications 
links 370-1 through 370-S. 

Servers 330-1 through 330-M are coupled to network 350 via connections 
332-1 through 332-M, respectively. Servers 330-1 through 330-M are coupled to 
the terrestrial links 360-1 through 360-T via antennae 334-1 through 334-M, 
respectively. Servers 330-1 through 330-M are coupled to space based 
communications links 370-1 through 370-S via dish antennae 336-1 through 



Clients 340-1 throu^ 340-N are connected to the network 350 via 
connections 342-1 through 342-N. Clients 340-1 through 340-N are connected 
to the terrestrial links 360-1 through 360-T via antennae 344-1 through 344-N. 
Clients 340-1 through 340-N are connected to space based communications 
links 370-1 through 370-S via dish antennae 346-1 through 346-N. Clients 340-1 
through 340-N may also be connected to web sites, search modules, and/or 
database resources represented by servers, such as servers 330-1 through 330- 
M, via the network 350, through connections 342-1 through 342-N. 
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Clients 340-1 through 340-N may include, but are not limited to, for 
example, a set-top box, a receiver, a television, a game platform, or other 
receiving devices. Applications may be running on the clients 340-1 through 
340-N, while web pages and information being browsed may reside on the 
servers 330-1 through 330-M. Broadcasts may be coming from terrestrial 
sources 360-1 through 360T, and/or satellite links 370-1 through 370-S. For 
purpose of explanation, a single client 340-1 will be considered to illustrate one 
embodiment of the present interactive techniques. It will be readily apparent 
that such interactive techniques may be applied easily applied to multiple 
clients. 

Network 350 may be a Wide Area Network (WAN), which includes the 
Internet, or other proprietary networks, such as America On-Line^, 
CompuServe™, Microsoft Network™, and Prodigy™. For other embodiments, 
network 350 may include Local Area Network (LAN), satellite link, fiber 
network, cable network, or any combination of the above. Network 350 may 
also include network backbones, long-haul telephone lineS/ Internet service 
providers, and various levels of network routers. 

Terrestrial links 360-1 through 36Q-T may be, for example, television 
broadcasters. Space based communications links 370-1 through 370-S may be, 
for example, satellite broadcasters. Communications network 300 may be 
implemented in any number of interactive environments. 

Figure 4 is an internal block diagram 400 of one embodiment of an 
application/game system, which may be representative of any of the clients 
and servers shown in Figure 3. Alternatively, the application/game system 
may be a stand alone unit such as a SONY Playstation™ or a personal 
computer. The following internal block diagram is a high level conceptual 
representation and may be implemented in a variety of ways and by various 
architectures. 

The system includes a bus 402 coupled to a Central Processing Unit 
(CPU) 404, Read Only Memory (ROM) 406, Random Access Memory (RAM) 
408, storage 410, display 420, audio 422, keyboard 424, pointer 426, 
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misceUaneous input/output (I/O) devices 428 coupled to respective I/O lines 
239/ and communication devices 430 coupled to respective external connection 
ports 432. 

Bus 402 may be, for example, one or more data busses such as a system 
bus. Peripheral Component Intercormect (PCI), Advanced Graphics Port 
(AGP), Small Computer System Interface (SCSI), Institute of Electrical and 
Electrorucs Modulers (IEEE) standard number 1394 (FireWire) or other like 
busses. 

CPU 404 may be a single, multiple, or even a distributed processing 
device. ROM 406 may be any type of non-volatile memory such as a mask 
programmable, flash, or other like memory. RAM 408 may be a static, 
dynamic, synchronous, or asynchronous memory. Storage 410 may be a 
Compact Disc (CD), Digital Video Disc pVD), hard disks, optical disks, tape, 
flash, memory sticks, video recorders, or other like storage devices. Display 
420 may be a Cathode Ray Tube (CRT), Liquid Crystal Display (LCD), a 
projection system. Television (TV), or other like display devices. Audio 422 
may be a monophonic, stereo, three dimensional sound card, or other like 
audio devices. 

Keyboard 424 may be a keyboard, a musical keyboard, a keypad, a 
series of switches, or other like input devices. Pointer 426 may be a mouse, a 
touchpad, a trackball, joystick, or other like pointing devices. I/O devices 428 
may be a voice command input device, a thumbprint input device, a smart card 
slot, a Personal Computer Card (PC Card) interface, virtual reality accessories, 
which may optionally connect via an input/ output port 429 to other devices or 
systems. An example of a miscellaneous I/O device 428 would be a Musical 
Instnmient Digital Interface (MIDI) card v^th the I/O port 429 connecting to 
the musical instnmient(s). Commimication devices 430 may be an Ethernet 
adapter for local area network (LAN) cormections, a satellite connection, a set- 
top box adapter, a Digital Subscriber Line (xDSL) adapter, a wireless modem, a 
conventional telephone modem, a direct telephone connection, a Hybrid-Fiber 
Coax (HFC) connection, cable modem, or other like devices. 
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External connection ports 432 may provide for any interconnection, as 
needed, between a remote device and bus 402 through the communication 
devices 430. For example, communications devices 430 may include an 
Ethernet adapter, which is connected via a connection port 432 such as an 
external DSL modem. Depending upon the actual implementation of the 
system, the system may include some, all, or more, or a recirrangement of 
components in block diagram 200. For example, a thin client may include a 
wireless hand held device that lacks, for example, a traditional keyboard. 
Thus, many variations on the system of Figure 4 are possible. 

Applications/game platform 103 of Figure 1 may operate as program 
instruction code stored in a memory device such as RAM 408. Alternatively, 
application/game platform 103 may operate as a separate processing system 
having internal components such as that shown in Figure 4. 

Figure 5 illustrates one embodiment of an entertainment system 
receiving audio/ video data with supplemental data. Referring to Figure 5, 
broadcaster 502 broadcasts A/ V data and supplemental data to DTV 512 via 
set-top box (STB) 510, which communicates to DTV 512 via a connection 511. 
Because of the capabilities of digital television, supplemental data may be 
provided and presented on DTV 512 with a broadcast of A/V data. For one 
embodiment, STB 510 separates supplemental data from the A/V data and 
delivers the A/V data to DTV 512. STB 510 may also process the supplemental 
data to generate visual enhancements on DTV 512. For an alternate 
embodiment, STB 510 may send the supplemental data to an external device, 
for example, a game console, which processes the supplemental data to 
generate rendered images for DTV 512 or for a remote display. 

Figure 6 illustrates one embodiment of various providers providing 
supplemental data with audio-video data (A/V data) for an entertainment 
system. Referring to Figure 6, broadcaster 602 provides A/V data, which is 
bundled with supplemental data from information providers 640. 

For one embodiment, information providers 640 may include a plurality 
of information providers such as race association 614, telemetry provider 618, 
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camera tracking provider 622, and provider N 626. Race association 614 may 
be a race sponsor such as CART™, which provides racecar standings data 616 
for broadcast 606. Racecar standings data may include status of a race car, 
driver, laps completed, and other like information. Telemetry provider 618 
provides racecar telemetry data 620 for broadcast 606. For the same race, a 
camera tracking provider 622 may provide camera tracking data 624 for 
broadcast 606. A plurality of providers may provide supplemental data such 
as content N data 628 from provider N 626 for broadcast 606. 

The A/V data 604 is combined or "btmdled" with supplemental data 640 
(standings data 616, telemetry data 620, tracking data 624, and content N data 
628) in broadcast 606 as a signal 608. Signal 608 is transmitted to a receiver or 
set-top box (STB) 610, which communicates via a connection 611 to digital TV 
(DTV) 612. For one embodiment, DTV 612 processes supplemental data 640 
bundled with A/V data 604 to provide visual enhancements. In alternate 
embodiment, STB 610 processes supplemental data 640 bundled with A/V data 
604 to provide visual enhancements. 

Standings data 616 may be presented to a user on DTV 612 as a visual 
enhancement Standings data 616 may include information related to position 
of a particular driver, best lap, and lap number. For example. Figure 12b 
illustrates an exemplary screen 1220 for DTV 612 showing a standings data 
visual enhancement. The A/V data for screen 1220 shows the racetrack and 
three race cars. Other visual enhancements 1222, 1223, and 1224 may be shown 
related to standings data. 

Telemetry data 620 may be presented to a user on DTV 612 as a visual 
enhancement. Telemetry data 620 may include information relating to a 
particular position of a race car, orientation, module revolutions per minute 
(RPM), gear of a race car, braking iirformation, transverse forces on a race car 
and other like information. For example. Figure 12a illtistrates an exemplary 
screen 1202 for DTV 612 showing telemetry data enhancements 1204, 1206, 
1208, 1210, and 1212. The main A/V data feed of screen 1202 shows the 
racetrack from a perspective of a camera positioned above a helmet of a driver 
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in the race car. Box display enhancement 1206 indicates the current RPM of the 
module of the race car, while enhancement 1208 displays the current speed. 
The shift pattern indicator enhancement 1210 indicates that the racecar is in 
fourth gear and indicator enhancement 1212 indicates that the driver is 



Tracking data 624 may be presented to a user on DTV 612 as a visual 
enhancement Tracking data 624 may include camera information relating to a 
location or the view it can provide, such as, a pit stop, aerial view, a particular 
tum in the racetrack, or a partioilar cockpit camera. The user may control 
which type of camera information to be shown. For example. Figure 12h 
illustrates an exemplary screen 1290 showing one camera view having a visual 
enhancement 1299 listing options for a user to select in which one of the 
options is for "cameras." That is, a user may use a remote control device to 
select the "cameras" option to choose a particular camera view. For example, a 
user may choose a cockpit view as shown in screen 1202 of Figure 12a or a 
different camera view as shown in screen 1220 of Figure 12b. 

Additional camera information may include camera orientation, field of 
view (zoom), radial distortion coefficients, or other like camera information. 
Camera orientation information may be useful, for example, to display to a 
viewer where cameras are located and the direction they are aimed. This 
information may allow a user to select a view of interest. Additionally, under 
automated control, it may be possible to ''track" a racecar as it moves around 
the racetrack by switching cameras to maintain the best view of the desired car. 
Alternatively, a user may request a graphical highlight be displayed to 
highlight a selected race car. Field of view information indicates to the viewer 
what may be viewed on a camera. Radial distortion coefficients may enable the 
viewing system to correct for errors. Additionally, the radial distortion 
coefficients may be of benefit in the case where a virtual racecar is on the track. 
With this radial distortion coefficient information, the rendering of the virtual 
racecar may be made more precise and/or realistic. 



braking. 
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Content N data 628 may represent other types of supplemental data to 
be used for displaying visual enhancements on DTV 612. For example, 
enhancement 1299 of Figure 12h shows an option for "replays." That is, 
information provider N 626 may provide "replay tags" for interesting events 
such as the start of the race, car crashes, and the finish. A user or viewer of 
DTV 616 may to the replay tags to view the tagged information or live video 
scenes. The tagged information may be stored in a local storage device or a 
remote storage device such as a database on the Internet. 

Content N data 626 may represent transverse force data. For example, 
as a racecar goes around a right turn, the transverse forces are attempting to 
push it to the left outside as indicated by, for example, enhancement arrow 
1204 in Figure 12a shows such a traiisverse force. Figure 12g illustrates a 
different presentation of visual enhancements based on provided supplemental 
ir\formation. The n\ain A/V data feed is shown in screen 1280. Superimposed 
are transverse force arrows 1282 and 1281 for two cars. On the left side of the 
screen is a presentation area 1283 containing additional information. Here the 
information is Performance 1284, the race is Road America, and the camera is at 
the "Carousel Turn" 1285. Thus, the main scene 1280 is of the "Carousel Turn." 
A vertical graph 1286 displays the Lateral Force 1289 experienced by the two 
cars 1287 and 1288 corresponding to the lateral force as indicated by the arrows 
1282 and 1281 in the main view 1202. 

Figure 7 illustrates one embodiment of a game platform integrated with 
an entertainment system capable of receiving audio/video data and 
supplemental data. Referring to Figure 7, game platform 730 is shown 
communicating via a liiJs 729 with broadcast receiver or set-top box (STB) 710. 
Alternatively, game platform 730 may be contained in STB 710. 

STB 710 may send broadcast 706 including A/V data 704 and 
supplemental data 740 to DTV 712 via channel 711. For one embodiment, STB 
710 sends A/V data to DTV 712 and supplemental data 740 to game platform 
730. In alternate embodiment, STB 710 sends both A/V data and supplemental 
data to game platform 730. Game platform 730 processes supplemental data 
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740 to generate game data, which may be sent to DTV 712 via STB 710 and 
connection 711. Thus, game platform 730 may interact with supplemental data 
740 to generate game elements or virtual objects for a game environment on 
DTV 712. Game platform 730 may be used with STB 710 or DTV 712 to provide 
the user with the ability to select options, camera views, replay, teams as 
described above. 

By having the supplemental data and A/V data feed available, it is 
possible for a user to manipulate the viewing environment or '"entertainment 
context" One such enhancement as described above, is the presentation of 
additional informahon on DTV 712. With the addition of game platform 730 
lirUced into receiver 710 and DTV 712, it is possible to pass live telemetry data 
from DTV 712 to game platform 730 and then pass live game data from game 
platform 730 to DTV 712. Game platform 730 may also be connected to the 
Internet to receive other types of information, which may be used with 
supplemental information 740 to generate game data. 

This capability allows for new modes of entertainment context. One 
mode is where game data is presented to DTV 712 with the A/V data in which 
use of supplemental data assists in the realism of the game. Another mode is 
where DTV 712 has primarily A/V data that is supplemented by game data 
from a game console. Furthermore, a combination is possible, which may 
change over time if necessary. 

An example of predominantly game data being presented aided with 
A/V data and supplemental data is shown in Figure 12f. An A/V data feed 
from a broadcaster is shown in screen 1260, which shows an actual racecar 1264 
on racetrack 1266. Screen 1270 shows a predominantly game generated scene 
with a virtual ("game") racecar 1272 on a racetrack 1276. A user or player may 
control virtual racecar 1272 based on the view of actual race car 1264 of screen 
1260. Thus, by using supplemental data, there is presented an overhead view 
of the upcoming track 1274 superimposed on the display. The "tracking" of the 
game created scene 1270 with the live A/V data scene may be achieved by 
having an accurate game representation of the racetrack locally, for example. 
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on a DVD, or a real-time conversion of the A/ V data 1260 into a game 
presentation 1270, or a combination. 

An example of the mode where the DTV has primarily A/ V data that is 
supplemented by game data from the game console is shown in Figure 12h. 
The main A/ V data feed is shown in screen 1290 showing real racecars 1294, 
1292, and 1292 on racetrack 1296. Race car 1298 may be, for example, a 
highlighted virtual car representation for a game player to compete with race 
cars of the actual race event. Panel 1299 is a visual enhancement providing 
various menu options, in this case, indicating that the Play mode is active. 
Thus, Figure 12h illustrates a game playing mode where the driver of the 
"game" or virtual car may participate in a predominantly real-time race scene 
1290 with real race cars 1296 and 1294. For an alternate embodiment, screen 
1290 may represent a rendered broadcast of the actual sports event in a virtual 
game world. That is, the objects in screen 1290 may be reconstructed in a 
rendered virtual gcune environment in which race car objects in the game 
envirorunent are rendered race cars from the actual broadcast 

Figure 8 illustrates another embodiment of a game platform integrated 
with an entertainment system capable of receiving audio/ video data and 
supplemental data from a broadcast and information from an auxiliary 
information source. Referring to Figure 8, an environment wherein a game 
platform 830 is shown commimicating via a link 829 with the broadcast 
receiver or set-top box (STB) 810. 

STB 810 may send broadcast 806 including A/ V data 804 and 
supplemental data 840 to DTV 812 via cormection 811. Additionally, STB 810 
may communicate with auxiliary information source 832 via 831. Link 831 may 
be, for example, a connection to the Internet. Auxiliary information source 832, 
may provide game information related to a game platform of a remote player 
through an IP connection. Thus, STB 810 may send game data from a remote 
player to game platform 830 and DTV 812. STB 810 may also send local game 
player data to auxiliary information source 832. 
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Game platform 830 may send game data to auxiliary information source 
832 via STB 810. STB 810 may send the game data from game platform to DTV 
812. In this way, game platform 830 may interact with supplemental data 840. 
Game platform 830 may be used in addition to, or in conjunction with, the STB 
810 and/or DTV 812 to provide the user with the ability to select options, 
camera views, replay, fantasy teams, and other options as described above. By 
having auxiliary information available from another remote game player via 
the auxiliary information source 832, it is possible for the local game player to 
share information and/or interact with the remote player to share a common 
viewing environment or "entertainment context/' 

One such possibility, in the racing car context, is to have multiple remote 
game players effectively represented on a local DTV for each player as if they 
were all in the actual race scene. 

Figure 9 is an internal block diagram of one embodiment of a set-top 
box 903. For one embodiment, set-top box 903 may be set-top 102 of Figure 1. 
Referring to Figure 9, set-top box 903 includes receiving module 952, 
compositor module 954, and application/game platform module 956. 
Receiving module 952, compositor module 954, and application/ game 
platform module 956 may represent hardware or software or a combination of 
both for set-top box 903. In an alternate embodimeiit, application/ game 
platform module may be hardware or software located outside of set-top box 
903. 

Receiving module 952 receives audio-video A/V data and supplemental 
data from a broadcaster. The A/V data may be digital or analog data. 
Receiving module 952 is responsible for de-multiplexing the A/V data and the 
supplemental data such that the audio video data is sent to compositor module 
954 and supplemental data is sent to application/game platform module 956. 

Application/game platform module 956 receives supplemental data 
from receiving module 952 and processes the supplemental data to generate 
application or game enhancement data. In alternate embodiment, 
application/game platform module 956 may receive both A/V data and 
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supplemental data. The application or game enhancement data is used to 
create visual enhancements on A/V data for display 901. Application/game 
platfomi module 956 may receive inputs from a user/player of a home 
network via remote control or a game console. Furthermore, applicaticm game 
platform may receive inputs from a remote user through one or more external 
connections 1 through N. For one embodiment, application/ game platform 
module 956 may generate enhancement data based on a type of interactive 
service requested from a user. 

Compositor module 954 receives A/V data from receiving module 952 
and enhancement data from application/ game platform module 956. 
Compositor module 954 is responsible for combining the A/V data with the 
enhancement data and sending the combined data to display 901. Display 901 
is configured to display the A/V data, for example, a live sports event, along 
with visual enhancements based on the enhancement data from application 
game platform module 956. 

Figure 10 is a functional block diagram of one embodiment of an 
application/game platform module 1056. For one embodiment, 
application /game platform module 1056 is a functional block diagram for 
application/ game platform module 956 of Figure 9. Referring to Figure 10, 
application/game platform 1056 includes a supplemental data processing 
module 1058 and an enhancement data processing module 1070. 

Supplemental data processing module 1058 receives the supplemental 
data and processes the supplemental data for enhancement data processing . 
module 1070. For example, the supplemental data may include a variety of 
types of supplemental data such as telemetry data, camera positioning data, 
standing data, or other types of data from various information provided. 
Supplemental data processing module 1058 is responsible processing the 
supplemental data to separate the different types of data for enhancement data 
processing module 1070. 
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Enhancement data processing module 1070 processes the data from 
supplemental data processing module 1058 based on specific requests from a 
user /player to generate enhancement data based on a user/player request. 
Enhancement data processing module 1070 may be coupled to a variety of 
extenul cormections such as the Internet, multiple players, a telephone line, or 
a cable satellite connection for receiving other types of information. For 
example, a user/player may receive game data from another player via an 
Internet connection to play in a multi-player game environment. Thus, 
enhancement data processing module 1070 is controlled by a user/player that 
controls which type of visual enhancements are to be displayed such as that 
shown in Figures 12a through 12h. 

Figure 11 illustrates one embodiment of an electronic program guide for 
allowing a user to access interactive services. Referring to Figure 11, a home 
environment is shown having display 1101 coupled to set-top box 1102 which 
is able to receive inputs from keypad 1116 on remote control 1119 and 
application/game platform 1103. 

A user/player may control remote control 1119 or application/game 
platform 1103 to access a program guide 1110. For example, a user may hit a 
key on remote control 1119 to access a program guide generator in STB 1102 or 
within display 1101. Program guide 1110 may provide a plurality of options 
such as interactive service 1112. A user/player may select interactive service 
1112 to access a number of different types of services as shown, for example, in 
window 1114. For example, the various types of services may include basic, 
enhanced games, and interactive games. 

If a user selects basic service, a user may be provided with a service to 
generate a visual enhancement of a virtual cockpit with virtual gauges of an 
actual cockpit of a race car of a selected driver in a live race event. The virtual 
cockpit is linked to telemetry data provided from the actual race car of the 
selected driver. For example, as shown in Figure 12a, A/V data of a cockpit 
view is shown in display 1202. Visual enhancements 1204, 1206, 1210, and 1208 
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may show telemetry data from the actual race car of a selected driver from race 
car broadcast. 

If a user selects enhanced games, a user may be provided with a service 
to generate visual enhancements such as that shown in Figures 12b, 12c, and 



Figure 12b illustrates screen 1220 for display 1101 showing visual 
enhancements of current standings and team performance. A user/viewer 
may request a visual enhancement to display current standings, lap times, or 
other real-time information for the race leaders or fantasy teams. Visucil 
enhancement 1222 shows options for viewing such as standings, best laps, and 
number and of pit stops. Visual enhancement 1223 shows options for cars such 
as leaders and team players that may be playing a fantasy team league 
competition. Visual enhancement 1224 may show the score of multiple players 
as they relate to the events shown in A/ V data 1220. 

Figure 12c illustrates screen 1230 for display 1101 showing visual 
enhancements of interactive polls. A user/ viewer my request a visual 
enhancement of trivia questions or questions related to events in the race that 
allow the viewer to earn additional points for a fantasy team league 
competition. A shown in Figure 12c, screen 1230 of a live A/ V data broadcast 
is shown with visual enhancements 1236, 1237, and 1238 allow a user/viewer 
to answer trivia questions. Visual enhancement 1236 shows the time remaining 
in answering a question. Visual enhancement 1237 display options for 
answers. Visual enhancement 1238 shows scores of multiple players capable of 
answering questions while viewing with A/V data broadcast 1238. 

Figure 12d illustrates screen 1240 for display 1101 showing visual 
enhancements of scoring for a fantasy team league competition. As shown in 
Figure 12d, an A/V data broadcast is displayed in screen 1240 along with 
visual enhancements 1241, 1242, and 1244. Visual enhancement 1241 shows a 
particular player time and points for each selected driver. Visual enhancement 
1242 shows a scoring table. Visual enhancement 1244 shows the score of 
multiple players in the fantasy team league. 



12d. 
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If a user selects interactive games, a user may be provided with a service 
to generate visual enhancements of a virtual race car on a live race car sports 
event broadcast as explained with reference to Figures 12f and 12h above. The 
player of the virtual race car may compete against the actual race car drivers of 
the broadcast. 

Figure 13 is a block diagram of one embodiment of an application/ game 
platform. As shown in Figiure 13, the application/ game platform 1300 includes 
a data interface 1310 for receiving supplemental data transmitted by a 
broadcaster (not shown), and a processing module 1320 for processing the 
supplemental data to make it suitable for integration. The platform 1300 
further includes a simulating module 1330 for receiving the processed 
supplemental data and for integrating the processed data into the game 
platform 1300/ such that the processed supplemental data is associated with 
one or more user controllable game elements generated by the game platform 



In one embodiment, supplemental data received in the game platform 
1300 further includes data collected by multiple sensor devices attached to 
participants to the broadcast event, for example participants engaged in a 
sports broadcast event. Alternatively, data collected by multiple cameras in the 
broadcast event may be included in the transmitted supplemental data. 

In one embodiment, the participants are live race cars engaged in a race 
car event. Alternatively, the participants may be other entities engaged in a 
different sports event. 

In one embodiment, supplemental data is transmitted by telemetry and 
synchronized to broadcast data, for example digital audio/video data, emitted 
by the broadcaster. Alternatively, supplemental data may be transmitted by 
other transmission means. In one embodiment, supplemental data includes 
race car variables, for example speed, direction, engine rotation per minute, 
gear, braking, lateral and longitudinal acceleration, and distance traveled. 
Alternatively, other variables may be included in the transmission of 



1300. 
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supplemental data. For example, in one embodiment, supplemental data may 
include camera data. 

In one embodiment, supplemental data is real-time operational data 
related to the participants to the broadcast event. In one embodiment, the 
supplemental data is transmitted live. Alternatively, supplemental data may 
be recorded and subsequently transmitted. 

Data interface 1310 receives the supplemental data and forwards it to 
the processing module 1320. Supplemental data is transmitted at a 
predetermined frequency, which may or may not coincide with a cycle time of 
the game platform 1300. Processing module 1320 monitors the frequency of the 
supplemental data and processes the supplemental data to obtain processed 
data. 

In one embodiment, the frequency is higher than the cycle time of the 
game platform 1300. Processing module 1320 receives the supplemental data 
and discards certain supplemental data at predetermined periods of time. 
Thien, processing module 1320 processes the remaining supplemental data to 
obtain processed supplemental data. In one embodiment, processing module 
1320 transforms the remaining supplemental data into interactive game 
representations of supplemental data in order to allow simulating module 1330 
to integrate the processed supplemental data with one or more game elements 
generated by the game platform 1300. 

Alternatively, the frequency at which supplemental data is transmitted 
may be lower than the cyde time of the game platform 1300. In this 
embodiment, processing module 1320 extrapolates the supplemental data to 
obtain predicted data and processes the predicted data to obtain processed 
data. In one embodiment, processing module 1320 uses a dead-reckoning 
process to extrapolate the supplemental data and to obtain predicted data. 
Alternatively, other methods of extrapolation may be used to obtain the 
predicted data. Subsequently, processing module 1320 transforms the 
predicted data into interactive game representations of supplemental data in 
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order to allow simulating module 1330 to integrate the processed supplemental 
data with one or more game elements generated by the game platform 1300. 

In another alternate embodiment, the frequency at which supplemental 
data is transmitted may be equal to the cycle time of the game platform 1300. 
In this embodiment processing module 1320 receives the supplemental data 
and proceeds to transform the supplemental data into interactive game 
representations. The interactive game representations are derived from the 
supplemental data in order to allow simulating module 1330 to process and 
integrate the supplemental data with one or more user controllable game 
elements generated by the game platform 1300. 

Simulating module 1330 receives the processed supplemental data and 
integrates the processed data with the game elements generated by the game 
platform 1300. In one embodiment, simulating module 1330 is an engine of the 
application/game platform 1300. Alternatively, simulating module 1330 may 
be implemented in hardware and included in the game platform 1300. 

In one embodiment, the interactive game representations of 
supplemental data, i.e. the processed supplemental data, include one 
participant game element corresponding to each participant in the broadcast 
event, each participant game element being rendered in coordinates similar to 
the user controllable game elements generated by the game platform 1300, 
which include representations of game elements controlled by users of the 
game platform 1300. 

Alternatively, the processed supplemental data may include 
audio/video data corresponding to each participant in the broadcast event. In 
this embodiment, processed supplemental data is further integrated with one 
or more user controllable game elements generated by the game platform 1300. 

In one embodiment, the broadcast event is a race car sports broadcast 
event, wherein supplemental data received from event race cars is processed 
and transformed into participant game elements of event race cars, which are 
integrated with user controllable simulated race cars belonging to the users of 
the game platform 1300. Alternatively, audio/video data corresponding to 
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each event race car is integrated with the user controllable simulated race cars 
generated by the game platform 1300. 

A procedure for integrating processed supplemental data with user 
controllable game elements generated by the platform will be described in 
further detail below in connection with virtual collision scenarios between 
participant game elements of event race cars and user controllable game 
element race cars. 

In one embodiment, if a virtual collision is detected between two 
participant game elements, for example representations of real race cars, the 
simulating module 1330 updates the positions of the two participant game 
elements using processed supplemental data. Alternatively, if a virtual 
collision is detected between two user controllable game elements, for example 
simulated race cars, the simulating module 1330 updates the positions of the 
user controllable game elements using internal artificial intelligence 
capabilities. 

In an alternate embodiment, if a user controUable game element 
virtually collides with one participant game element, the simulating module 
1330 addresses the virtual collisions based on the game element that provoked 
them. 

If a user controllable game element provokes the virtual collision, 
simulating module 1330 detennines the effects on the user controllable game 
element using internal artificial intelligence capabilities. The effects on the 
participant game element depend on the use of processed supplemental data. 
In one embodiment, the simulating module 1330 uses only processed 
supplemental data to position the participant game elements. As a result, 
simulating module 1330 will not corwider the effect of the collision on the 
position of the participant game element involved in the collision, and will 
update the position using only the processed supplemental data. Alternatively, 
simulating module 1330 may use processed supplemental data and internal 
artificial intelligence capabilities to calculate a new position of the participant 
game element. Simulating module 1330 may modify the position of the 
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participant game element to the new position, and subsequently may gradually 
update any subsequent position using the processed supplemental data. 

If a participant game element provokes the virtual collision, the 
simulating module 1330 determines the effects on the user controllable game 
element using its internal artificial intelligence capabilities. In one 
embodiment, simulating module 1330 updates the position of the user 
controllable game element by maintaining the current position without taking 
into account the virtual collision. Alternatively, simulating module 1330 may 
calculate a new position of the user controllable game element and may update 
the position of the user controllable game element to the new position. 

Figure 14 is a flow diagram of the method for leveraging data into a 
game platform. Referring to Figure 14, at processing block 1410, supplemental 
data related to a broadcast event is received. At processing block 1420, 
supplemental data is processed to obtain processed data. Finally, at processing 
block 1430, processed data is integrated into a game platform, such that the 
processed data is associated with one or more user controllable game elements 
generated by the game platform. 

Figure 15 is a flow diagram of one embodiment of a procedure for 
processing supplemental data. Referring to Figure 15, once supplemental data 
is received, at processing block 1510, a decision is made whether the 
predetermined frequency at which supplemental data is transmitted is higher 
then a cycle time of the game platform. If the frequency is higher than the cycle 
time, at processing block 1520, supplemental data is discarded at 
predetermined time periods. Next, at processing block 1530, remaining 
supplemental data is processed to obtain processed data. 

Otherwise, if the frequency is not higher then the cycle time, at 
processing block 1540, a decision is made whether the frequency is lower than 
the cycle time. If the frequency is lower than the cyde time, at processing block 
1550, supplemental data is extrapolated to obtain predicted data. Next, at 
processing block 1560, predicted data is processed to obtain processed data. 
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If the frequency is not lower than the cycle time of the game platform, 
then^ at processing block 1570, supplemental data received is transformed into 
processed data. Finally, at processing block 1580, processed data is forwarded 
to simulating module 1330. 

Figure 16 is a flow diagram of one embodiment of a procedure for 
integrating processed data into the game platform. As shown in the flow 
diagram of Figiu-e 16, at processing block 1610, processed data is received. In 
one embodiment, processed data is received from the processing module 1320. 
At processing block 1620, a decision is made whether a user controllable game 
element provokes a virtual collision with a participant game element. If yes, in 
one embodiment, a new position of the participant game element is calculated 
at processing block 1630. At processing block 1640, the position of the 
participant game element is modified to the new position calculated. Finally, at 
processing block 1650, the position of the participant game element is updated 
using subsequent processed supplemental data. In an alternate embodiment, if 
the user controllable game element provokes the virtual collision with the 
participant game element, the position of the participant game element may be 
updated using processed supplemental data at processing block 1650, 
irrespective of any new calculated position. 

If the user controllable game element does not provoke the virtual 
collision with the partidpemt game element, at processing block 1660, a 
decision is made whether the participant game element provokes the virtual 
collision with the user controllable game element. If the participant game 
element provokes the virtual collision, at processing block 1670, a new position 
of the user controllable game element is calculated. Then, at processing block 
1680, the position of the user controllable game element is updated. In one 
embodiment, the position of the user controllable game element is updated by 
maintaining the position without relying on the calculated new position. 
Alternatively, the position of the user controllable game element may be 
updated to the calculated new position. 
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If the virtual collision is not provoked by the participant game element, 
at processing block 1690, the positions of the user controllable game element 
and the participant game element are updated using processed supplemental 
data. 

It is to be understood that embodiments of this invention may be used as 
or to support software programs executed upon some form of processing core 
(such as the CPU of a computer) or otherwise implemented or realized upon or 
within a machine readable medium. A machine readable medium includes any 
mechanism for storing or transmitting information in a form readable by a 
machine (e.g., a computer). For example, a machine readable medium includes 
read-only memory (ROM); random access memory (RAM); magnetic disk 
storage media; optical storage media; flash memory devices; electrical, optical, 
acoustical or other form of propagated signals (e.g., carrier waves, infrared 
sigrmls, digital signals, etc.); or any other type of media suitable for storing or 
transmitting information. 

In the foregoing specification the invention has been described with 
reference to specific exemplary embodiments thereof. It will, however, be 
evident that various modifications and changes may be made thereto without 
departing firom broader spirit and scope of the invention as set forth in the 
appended claims. The specification and drawings are, accordingly, to be 
regarded in an illustrative rather a restrictive sense. 
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CLAIMS 

What is claimed is: 

1. A method comprising: 

receiving supplemental data related to a broadcast event (1410); 
processing selectively said supplemental data to obtain processed data 
(1420); and 

integrating said processed data into a game platform, such that said 
processed data is associated with at least one user controllable game element 
generated by said game platform (1430). 

2. The method according to claim 1, wherein said receiving further 
comprises receiving supplemental data related to a sports broadcast event 



3. The method according to claim 1, wherein said receiving further 
comprises receiving real-time operational data related to a plurality of 
participants in said broadcast event (1410). 

4. The method according to claim 3, wherein said real-time 
operational data further includes telemetry data. 

5. The method according to claim 3, wherein said real-time 
operational data further includes camera data. 

6. The method according to claim 3, wherein said real-time 
operational data further includes data captured by a plurality of sensor devices 
attached to each participant of said plurality of participants in said broadcast 
event. 



(1410). 
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7. The method according to claim 3, wherein said real-time 
operational data further includes data captured by a plurality of cameras in 
said broadcast event. 

8. The method according to claim 1, wherein said receiving further 
comprises receiving said supplemental data S3mchronized with broadcast data 
(1410). 

9. The method according to claim 1, wherein said receiving further 
comprises receiving said supplemental data online using a modem (1410). 

10. The method according to claim 1, wherein said receiving further 
comprises receiving said supplemental data from a broadcast module through 
a data interface (1410). 

11. The method according to claim 1, wherein said receiving further 
comprises receiving said supplemental data at a predetermined frequency 
(1410). 

12. The method according to claim 11, wherein said processing 
further comprises: 

discarding said supplemental data at a predetermined time period, if 
said predetermined frequency is' higher than a cycle time of said game platform 
(1520); and 

processing remairung supplemental data to obtain said processed data 

(1530). 

13. The method according to claim 11, wherein said processing 
further comprises transforming said supplemental data into processed data, if 
said predetermined frequency is equal to a cycle time of said interactive service 
(1570). 

37 



Patent provided by Sughrue MIon, PLLC - http://www.sughrue.com 



wo 01/36061 



PCTAJSOO/30847 



14. The method according to claim 11, wherein said processing 
further comprises: 

extrapolating said supplemental data into predicted data, if said 
predetermined frequency is lower than a cycle time of said interactive service 
(1550); and 

processing said predicted data to obtain said processed data (1560). 

15. The method according to claim 8, wherein said broadcast data 
further includes digital audio/video data. 

16. The method according to claim 1, wherein said receiving further 
comprises receiving recorded supplemental data of a plurality of participants 
in said broadcast event (1410). 

17. The method according to claim 1, wherein said integrating 
further comprises updating a position of a participant game element in said 
game platform using said processed data, said participant game element 
corresponding to each participant of a plurality of participants in said 
broadcast event (1690). 

18. The method according to claim 1, wherein said integrating 
further comprises updating a position of a participant game element in said 
game platform, said participant game element corresponding to one 
participant of a plurality of participants in said broadcast event, if said at least 
one user controllable game element virtually collides with said participant 
game element. 

19. The method according to claim 18, wherein said updating further 

comprises: 
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calculating a new position of said participant game element in said game 
platform using said processed data (1630); 

modifjring said position of said participant game element to said new 
position within said game platform (1640); and 

updating said new position of said participant game element using said 
processed data (1650). 

20. The method according to claim 18, wherein said updating further 
comprises: 

updating said position of said participant game element using said 
processed data (1650). 

21. The method according to claim 1, wherein said integrating 
further comprises updating a position of said at least one user controllable 
game element in said game platform, if a participant game element 
corresponding to one participant of a plurality of participants in said broadcast 
event virtually collides with said at least one user controllable game element 

22. The method according to claim 21, wherein said updating further 
comprises: 

calculating a new position of said at least one user controllable game 
element (1670); and 

updating said position of said at least one user controllable game 
element to said new position (1680). 

23. The method according to daim 21, wherein said updating further 
comprises maintaining said position of said at least one user controllable game 
element (1680). 
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24. The method according to claim 1, wherein said supplemental data 
further includes audio/ video data corresponding to each participant of a 
plurality of participants in said broadcast event. 

25. The method according to claim 24, wherein said integrating 
further comprises integrating said audio/video data with said at least one user 
controllable game element generated by said game platform (1430). 

26. A system comprising: 

a processing module (1320) for processing supplemental data related to 
a broadcast event to obtain processed data (1420); and 

a simulating module (1330) for integrating said processed data into a 
game platform (1430), such that said processed data is associated with at least 
one user controllable game element generated by said game platform (1300). 

27. The system according to claim 26, further comprising a data 
interface (1310) for receiving said supplemental data (1410). 

28. The system according to claim 26, wherein said supplemental 
data is related to a sports broadcast event. 

29. The system according to claim 26, wherein said supplemental 
data is real-time operational data related to a plurality of participants in said 
broadcast event. 

30. The system according to claim 29, wherein said real-time 
operational data further includes telemetry data. 

31 . The system according to claim 29, wherein said real-time 
operational data further includes camera data. 
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32. The system according to claim 29, wherein said real-time 
operational data further includes data captured by a plurality of sensor devices 
attached to each participant of said plurality of participants in said broadcast 
event. 

33. The system according to claim 29, wherein said real-time 
operational data further includes data captured by a plurality of cameras in 
said broadcast event. 

34. The system according to claim 27, wherein said data interface 
(1310) receives said supplemental data synchronized with broadcast data. 

35. The system according to claim 27, wherein said data interface 
(1310) is a modem for receiving said supplemental data online. 

36. The system according to claim 27, wherein said data interface 
(1310) receives said supplemental data at a predetermined frequency. 

37. The system according to daim 36, wherein said processing 
module (1320) further discards said supplemental data at a predetermined time 
period, if said predetermined frequency is higher than a cycle time of said 
interactive service (1520), and processes remaining supplemental data to obtain 
said processed data (1530). 

38. The system according to daim 36, wherein said processing 
module (1320) further transforms said supplemental data into processed data, 
if said predetermined frequency is equal to a cyde time of said interactive 
service (1570). 



39. The system according to claim 36, wherein said processing 
module (1320) further extrapolates said supplemental data into predicted data, 
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if said predetermined frequency is lower than a cycle time of said interactive 
service (1550), and processes said predicted data to obtain said processed data 



40. The system according to claim 34, wherein said broadcast data 
further indudes digital audio/video data. 

41. The system according to claim 26, wherein said data interface 
(1310) receives delayed replay operational data of a plurality of participants in 
said broadcast event. 

42. The system according to claim 26, wherein said simulating 
module (1330) further updates a position of a participant game element in said 
game platform using said processed data, said participant game element 
corresponding to each participant of a plurality of participants in said 
broadcast event (1690). 

43. The system according to claim 26, wherein said simulating 
module (1330) further updates a position of a participant game element in said 
game platform, said participant game element corresponding to one 
participant of a plurality of participants in said broadcast event, if said at least 
one user controllable game element virtually collides with said participant 
game element. 

44. The system according to claim 43, wherein said simulating 
module (1330) further: 

calculates a new position of said participant game element in said game 
platform using said processed data (1630); 

modifies said position of said participant game element to said new 
position within said game platform (1640); and 



(1560). 
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updates said new position of said participant game element using said 
processed data (1650). 

45. The system according to claim 43, wherein said simulating 
module (1330) further updates Sciid position of said participant game element 
using said processed data (1650). 

46. The system according to claim 26, wherein said simulating 
module (1330) further updates a position of said at least one user controllable 
game element in said game platform, if a participant game element 
corresponding to one participant of a plurality of participants in said broadcast 
event virtually collides with at least one user controllable game element. 

47. The system according to claim 46, wherein said simulating 
module (1330) further calculates a new position of said at least one user 
controllable game element (1670), and updates said position of said at least one 
user controllable game element to said new position (1680). 

48. The system according to claim 46, wherein said simulating 
module (1330) further maintains said position of said at least one user 
controllable game element (1680). 

49. The system according to claim 26, wherein said supplemental 
data further includes audio/video data corresponding to each participant of a 
plurality of participants in said broadcast event. 

50. The system according to claim 49, wherein said simulating 
module (1330) further integrates said audio/video data with said at least one 
user controllable game element generated by said game platform (1430). 
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51 . A cx)mputer readable medium containing executable instructions 
which, when executed in a processing system, cause the system to perform a 
method comprising: 

receiving supplemental data related to a broadcast event (1410); 
processing selectively said supplemental data to obtain processed data 
(1420); and 

integrating said processed data into a game platform, such that said 
processed data is associated with at least one user controllable game element 
generated by said game platform (1430). 

52. The computer readable medium according to claim 51, wherein 
said receiving further comprises receiving supplemental data related to a 
sports broadcast event (1410). 

53. The computer readable meditun according to claim 51, wherein 
said receiving further comprises receiving real-time operational data related to 
a plurality of participants in said broadcast event (1410). 

54. The computer readable medium according to claim 53, wherein 
said real-time operational data further includes telemetry data. 

55. The computer readable medium according to claim 53, wherein 
said real-time operational data further includes camera data. 

56. The computer readable medium according to claim 53, wherein 
said real-time operational data further includes data captured by a plurality of 
sensor devices attached to each participant of said plurality of participants in 
said broadcast event. 
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57. The computer readable medium according to claim 53/ wherein 
said real-time operational data further includes data captured by a plurality of 
cameras in said broadcast event. 

58. The computer readable mediimi according to claim 51, wherein 
said receiving further comprises receiving said supplemental data 
synchronized with broadcast data (1410). 

59. The computer readable medium according to claim 51, wherein 
said receiving further comprises receiving said supplemental data online using 
a modem (1410). 

60. The computer readable medium according to claim 51, wherein 
said receiving further comprises receiving said supplemental data from a 
broadcast module through a data interface (1410). 

61. The computer readable medium according to claim 51, wherein 
said receiving further comprises receiving said supplemental data at a 
predetermined frequency (1410). 

62. The computer readable medium according to claim 61, wherein 
said processing further comprises: 

discarding said supplemental data at a predetermined time period, if 
said predetermined frequency is higher than a cycle time of said game platform 



63. The computer readable medium according to daim 61, wherein 
said processing further comprises transforming said supplemental data into 



(1520); and 



processing remaiiung supplemental data to obtain said processed data 



(1530). 
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processed data, if said predetermined frequency is equal to a cycle time of said 
interactive service (1570). 

64. The computer readable medium according to claim 61, wherein 
said processing further comprises: 

extrapolating said supplemental data into predicted data, if said - 
predetermined frequency is lower than a cycle time of said interactive service 
(1550); and 

processing said predicted data to obtain said processed data (1560). 

65. The computer readable medium according to claim 58, wherein 
said broadcast data further includes digital audio/ video data. 

66. The computer readable medium according to claim 51, wherein 
said receiving further comprises receiving recorded supplemental data of a 
plurality of participants in said broadcast event (1410). 

67. The computer readable medium according to claim 51, wherein 
said integrating further comprises updating a position of a participant game 
element in said game platform using said processed data, said participant game 
element corresponding to each participant of a plurality of participants in said 
broadcast event (1690). 

68. The computer readable medium according to claim 51, wherein 
said integrating further comprises updating a position of a participant game 
element in said game platform, said participant game element corresponding to 
one participant of a plurality of participants in said broadcast event, if said at 
least one user controllable game element virtually collides with said participant 
game element 
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69. The computer readable medium according to claim 68, wherein 
said updating hxrther comprises: 

calculating a new position of said participant game element in said game 
platform using said processed data (1630); 

modifying said position of said participant game element to said new 
position within said game platform (1640); and 

updating said new position of said participant game element using said 
processed data (1650). 

70. The computer readable medium according to claim 68, wherein 
said updating further comprises: 

updating said position of said participant game element using said 
processed data (1650). 

71- The computer readable medium according to claim 51, wherein 
said integrating further comprises updating a position of said at least one user 
controllable game element in said game platform, if a participant game element 
corresponding to one participant of a plurality of participants in said broadcast 
event virtually collides with said at least one user controllable game element. 

72. The computer readable medium according to claim 71, wherein 
said updating further comprises: 

calculating a new position of said at least one user controllable game 
element (1670); and 

updating said position of said at least one user controllable game 
element to said new position (1680). 

73. The computer readable medium according to claim 71, wherein 
said updating further comprises maintaining said position of said at least one 
user controllable game element. 
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74. The computer readable medium according to claim 51, wherein 
said supplemental data hirther includes audio/video data corresponding to 
each participant of a plurality of participants in said broadcast event. 

75. The computer readable medium according to claim 74, wherein 
said integrating further comprises integrating said audio/video data with said 
at least one user controllable game element generated by said game platform 



(1430). 
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